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Office Action Summarv 


Application No. 

10/603,687 


Applicant(s) 

SIENEL ETAL 


Examiner 

Kristina B. Honeycutt 


Art Unit 

2178 





r/ie MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS. 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of tinne may be available under the provisions of 37 CPR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)^ Responsive to communication(s) filed on 26 June 2003 . 
2a)\3 This action is FINAL. 2b)l3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Exparfe Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) 1^ Claim(s) 1-10 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) S Claim(s) 1-10 is/are rejected. 
?)□ Claim(s) Is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 26 June 2003 is/are: a)[3 accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or fonm PTO-1 52. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19{a)-(d) or (f). 
aM All b)n Some * c)\J None of: 

1 .1^ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) S Notice of References Cited (PTO-892) 4) □ Inten^iew Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. ^ . 

3) 13 Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Informal Patent Application (PTO-152) 

Paper No(s)/Mall Date 6/26/03 . 6) □ Other: . 

U.S. Patent and Trademark Office 

PTOL-326 (Rev. 7-05) Office Action Summary Part of Paper No7Mail Date 1 01105 
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DETAILED ACTION 

1 . This action is responsive to communications: Application filed June 26, 2003 
with acknowledged foreign priority date August 5, 2002; Information Disclosure 
Statement filed June 26, 2003. 

This action is made Non-FinaL 

2. Claims 1-10 are pending in the case. Claims 1, 7, 9 and 10 are independent 
claims. 

Priority 

3. Acknowledgment is made of applicant's claim for foreign priority under 35 
U.S.C. 1 19(aHd). The certified copy has been filed in parent Application No. 
10/603687, filed on June 26, 2003. 

Information Disclosure Statement 

4. The information disclosure statement (IDS) was submitted on June 26, 2003. 
The submission is in compliance with the provisions of 37 CFR 1 .97. Accordingly, the 
information disclosure statement is being considered by the examiner. 
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Drawings 



5. The drawings filed on June 26, 2003 are accepted. 



Claim Rejections - 35 USC § 112 

The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

6. Claim 6 is rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. 

7. Claim 6 recites the limitation "the handling" in lines 1-2. There is insufficient 
antecedent basis for this limitation in the claim. 



Claim Rejections - 35 USC § 102 



The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the Invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
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351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21 (2) 
of such treaty in the English language. 

9. Claims 1-7, 9 arid 10 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Itoh et al. (EP 1 139335 A2; date of publication October 4, 2001 ; filed March 31 , 2000; 
priority claimed to JP 2000099418 filed March 31, 2000). 

Regarding independent claim 1, Itoh discloses a method for multimodal interaction 
between on user and a terminal comprising an input unit, an output unit and a browser 
application, which is capable of interpreting a mark-up language, wherein the interaction 
takes place over the input unit, and/or the output unit and by using the browser 
application and the mark-up language, and wherein, the mark-up language comprises 
an extension of multimodal meta tags for multimodal interactions, and wherein the 
method comprises the steps of: 

• interpreting multimodal meta tags from the mark-up language for controlling the 
multimodal interactions (Fig. 3; col. 7, para. 32; col. 8, para. 42-44; col. 9, para. 
48, 49 - as demonstrated in the figure and cited text, meta tags are interpreted 
for controlling the interactions since Itoh teaches analyzing tags in the HTML 
data in order to control input and output between the user and the terminal) and 

• processing data of the multimodal interaction with respect to the multimodal meta 
tags by using at least one input and/or output processing application (col. 7, para. 
33, 34; col. 8, para. 38; col. 9, para. 50, 51 , 53 - as demonstrated in the cited 
text, data is processed with respect to the meta tags using input and output 
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applications since Itoh teaches outputting voice data as requested according to 
the HTML meta tags). 

Regarding dependent claim 2, Itoh discloses the method according to claim 1, 
wherein: 

• the browser application is controlled corresponding to said multimodal meta tags 
and/or a communication between the browser application and the input and/or 
output processing applications is controlled corresponding to said multimodal 
meta tags (col. 7, para. 33, 34; col. 9, para. 49 - as demonstrated in the cited 
text, the browser application is controlled according to meta tags and 
communication between the input and output applications since Itoh teaches 
displaying information on the browser based on the HTML meta tags that are 
input by the user). 

Regarding dependent claim 3, Itoh discloses the method according to claim 1 , 
wherein: 

• said at least one input and/or output processing application is one of the following 
applications: 

- handwriting recognition application 

- speech recognition application 

- eye movement recognition application 

- speech generation application 
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(col. 6, para. 30 - as demonstrated in the cited text, input and/or output processing 
applications are speech recognition applications). 

Regarding dependent claim 4, Itoh discloses the method according to claim 1 , 
wherein: 

• said input and/or output processing application is provided at the terminal (col. 6, 
para. 30 - as demonstrated in the cited text, input and output applications are 
provided at the terminal). 

Regarding dependent claim 5, Itoh discloses the method according to claim 1, 
wherein: 

• said input and/or output processing application is realized as an application 
having a distributed architecture (col. 7, para. 36, 37 - as demonstrated in the 
cited text, input and output applications have distributed architecture since Itoh 
teaches processing occurring at both the terminal and remotely at the server). 

Regarding dependent claim 6, Itoh discloses the method according to claim 1, 
comprising: 

• by rules determining the handling of a plurality of multimodal interactions being 
related to each other (col. 10, para. 61 , 62 - as demonstrated in the cited text, 
rules determine the handling of a plurality of interactions being related since Itoh 
teaches processing multiple requests in sequential order). 
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Regarding independent claims 7, 9 and 10, the claims reflect the terminal, browser 
application and mark-up language for performing the operations of claim 1 and are 
rejected along the same rationale. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

10. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Itoh in 
view of Daniel (U.S. Patent 6597280; date of patent July 22, 2003; filed March 17, 
1997). 



Regarding dependent claim 8, Itoh does not disclose determining and evaluating time 
information of said multimodal interaction. Daniel teaches evaluating time information 
(col. 2, lines 29-33). It would have been obvious to one of ordinary skill in the art, 
having the teachings of Itoh and Daniel before him at the time the invention was made, 
to modify the terminal taught by Itoh to include evaluating time information as taught by 
Daniel, because evaluating time information, as taught by Daniel (col. 2, lines 29-33), 
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would allow users to determine the most efficient means for interacting with the 
terminal. 

Conclusion 

11 . The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Highlighting tool for search specification in a user interface of a computer system 
(U.S. Patent 5982370), 

• System and method for concurrent multimodal communication session 
persistence (U.S. Patent 6912581). 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kristina B. Honeycutt whose telephone number is 571- 
272-4123. The examiner can normally be reached on 8:00 am - 5:00 pm Monday- 
Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Stephen Hong can be reached on 571-272-4124. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
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Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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